Papers with inference network
Iterative Refinement in the Continuous Space for Non-Autoregressive Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing non-autoregressive inference procedures that refine in token space often require computational overhead. |
| Approach: | They propose an efficient inference procedure that iteratively refines translation purely in the continuous space using a latent variable instead of the latent variables. |
| Outcome: | The proposed procedure is twice as efficient and more effective than the existing EM-like inference procedure. |
Unsupervised Recurrent Neural Network Grammars (N19-1)
Copied to clipboard
| Challenge: | RNNGs model syntax and structure by incrementally generating a syntax tree and sentence in a top-down, left-to-right order. |
| Approach: | They explore unsupervised learning of recurrent neural network grammars for language modeling and grammar induction. |
| Outcome: | The proposed model outperforms standard sequential language models and improves parsing performance. |
ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translation (2020.acl-main)
Copied to clipboard
| Challenge: | a non-autoregressive machine translation model can minimize the autoregressive teacher's energy . engINE is an inference network trained to minimize the teacher' energy based on distilled corpora . |
| Approach: | They propose to train a non-autoregressive machine translation model to minimize autoregressive teacher energy by using an inference network instead of distilled corpora. |
| Outcome: | The proposed model achieves state-of-the-art non-autoregressive results on two datasets . the proposed model is trained to minimize the autoregressive teacher energy . |
A Probabilistic Generative Model for Typographical Analysis of Early Modern Printing (2020.acl-main)
Copied to clipboard
| Challenge: | Scholars often need to go beyond textual analysis for establishing provenance of historical documents. |
| Approach: | They propose a deep and interpretable probabilistic generative model to analyze glyph shapes in printed Early Modern documents by generating a latent vector responsible for inking variations, jitter, noise and other unforeseen phenomena. |
| Outcome: | The proposed model outperforms interpretable clustering baselines and overly-flexible deep generative models on the task of completely unsupervised discovery of typefaces in mixed-fonts documents. |
Narrative Text Generation with a Latent Discrete Plan (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on story generation has focused on generating consistent stories via story outlines using keywords or key phrases. |
| Approach: | They propose a deep latent variable model that first samples a sequence of anchor words, one per sentence in the story, as part of its generative process. |
| Outcome: | The proposed model gets favorable scores when evaluated on perplexity, diversity, and control of story via discrete plan. |
Benchmarking Approximate Inference Methods for Neural Structured Prediction (N19-1)
Copied to clipboard
| Challenge: | Structured prediction models often involve complex inference problems for which finding exact solutions is intractable. |
| Approach: | They propose to perform gradient descent with respect to the output structure directly and train a neural network to perform inference. |
| Outcome: | The proposed methods achieve better speed/accuracy/search error trade-off than gradient descent while being faster than exact inference at similar accuracy levels. |
NeuroMax: Enhancing Neural Topic Modeling via Maximizing Mutual Information and Group Topic Regularization (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in neural topic models have focused on two main directions: the integration of the inference network with a pre-trained language model and the modeling of the relationship between words and topics in the generative model. |
| Approach: | They propose a framework to maximize mutual information between topics and PLMs . Experimental results indicate that NeuroMax reduces inference time . |
| Outcome: | The proposed framework reduces inference time, generates more coherent topics and topic groups, and produces more representative document embeddings. |